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A  series  of  experiments  addressed  the  adequacy  of  act  generation 
performance ,  an  important  precursor  to  problem  structuring.  Each  of  two 
decision  problems  was  studied  by  a  series  of  three  experiments.  In  the 
first  experiment,  subjects  were  given  a  realistic  decision  problem  and 
asxed  to  respond  with  any  act  occurring  to  them.  In  the  second  experiment, 
the  acts  suggested  were  evaluated  by  different  subjects  for  feasibility. 
In  a  third  experiment,  additional  subjects  estimated  the  utility  of  the 
acts  judged  feasible.  The  act  generation  performance  of  subjects  was 
evaluated  using  two  techniques.  First,  a  decision  tree  was  generated  by 
the  experimenters  by  combining  the  acts  suggested  by  all  subjects.  The 
decision  tree  generated  by  each  subject  was  compared  with  the 
experimenter-generated  tree.  It  was  found  that  subjects  failed  to  generate 
important  limbs  and  branches  of  the  group  decision  tree.  Second,  the 
quality  of  the  trees  generated  by  individual  subjects  was  evaluated  by  an 
opportunity  loss  calculation.  This  calculation  provided  an  estimate  of  the 
potential  cost  of  failing  to  generate  limbs  and  branches  of  the  decision 
tree.  The  opportunity  loss  analysis  suggested  that  the  failure  to  generate 
a  complete  tree  could  be  costly. 


An  Evaluation  of  Hunan  Act  Generation  Performance 

Several  specific  processes  occur  prior  to  making  a  carefully-considered 
decision.  First,  a  decision  maker  must  recognize  that  a  problem  exists  for 
which  a  decision  is  required.  Next,  relevant  components  or  dimensions  of 
the  problem  are  identified.  These  components  include  1)  acts  which  may 
potentially  lead  to  the  goal,  2)  states  of  the  world  (hypotheses)  which 
may  influence  the  outcomes  of  these  acts,  and  3)  possible  consequences  or 
outcomes  of  acts.  These  processes,  and  others,  are  included  in  the  general 
term  "predecision  processes. "  Predecision  processes  are  important 
precursors  to  the  actual  decision,  and  can  have  a  major  impact  on  the 
quality  of  the  decision  (Humphreys,  1979) . 

In  this  paper  we  concentrate  on  one  of  the  predecison  processes,  act 
generation.  Act  generation  is  the  process  of  generating  options  for  action 
available  to  the  decision  maker.  These  acts,  designed  to  achieve  the 
desired  goal,  are  the  "menu"  of  options  from  which  the  decision  maker  must 
eventually  choose.  It  is  quite  important  that  this  menu  be  as  complete  as 
possible,  to  offer  the  decision  maker  an  effective  course  of  action.  If 
the  list  of  acts  generated  by  the  decision  maker  is  incomplete,  the 
subsequent  decision  may  not  be  the  best  which  could  be  made  under  the 
circumstances,  leading  to  possible  "opportunity  loss." 

The  investigation  of  human  act  generation  capablilites  has  received  scant 
attention  to  date.  The  lack  of  concern  with  act  generation,  and 
predecision  processes  in  general,  is  probably  due  to  several  historical 
factors.  These  topics  initially  appeared  to  be  intractable  for  several 
reasons.  First,  early  research  on  hunan  decision  making  compared  human 
performance  with  optimal  models.  As  there  were  no  optimal  models  for 
predecisicn  processes,  the  topic  was  largely  ignored.  Raiffa  (1968) ,  for 
example,  stated  that  elicitation  of  structure  was  an  important  problem  he 
"wanted  to  duck."  The  recent  emergence  of  a  cognitive  approach  to  decision 
making  (Wallsten,  1980)  has  broadened  the  areas  of  inquiry  to  include  All 
aspects  of  decision  making,  not  just  tractable  candidates  for  normative 
modeling.  Second,  even  though  predecision  processes  are  legitimate 
candidates  for  study  from  a  cognitive  perspective,  the  processes 
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themselves  appeared  to  be  intractable  topics  of  study  which  would  not 
reveal  their  secrets  easily. 

There  has  been  some  research  on  hypothesis  generation,  which  is  a  process 
relevant  to  act  generation  (Gettys  and  Fisher,  1979;  Manning,  Gettys, 
Nicewander,  Fisher,  &  Mehle,  1980;  Mehle,  Gettys,  Manning,  Baca  &  Fisher, 
in  press) .  The  major  conclusion  of  these  and  other  studies  was  that 
hypothesis  generation  performance  is  largely  dependant  on  the  success  of 
accessing  information  available  in  memory.  Subjects  failed  to  retrieve 
many  important  hypotheses  from  memory,  yet  when  asked  to  assess  the 
completeness  of  their  retrieval,  they  grossly  over-estimated  it  (Mehle, 
et.  al. ,  in  press) .  Hus  series  of  studies  suggests  similar  deficiencies 
in  accessing  acts  stored  in  memory  will  be  found,  due  to  the  similarity  of 
act  and  hypothesis  generation. 

Pits,  Sadis  and  Heerboth  (1980)  investigated  methods  of  eliciting  acts 
from  subjects  in  the  first  study  on  the  topic  of  act  generation  in  a 
decision  theory  context.  They  found  serial  presentation  of  problem 
objectives  or  goals  enhanced  act  generation  performance  as  compared  to 
conditions  where  subjects  were  given  either  exemplar  acts  or  all  problem 
objectives  simultaneously.  They  also  observed  that  subjects  did  not  seem 
to  generate  a  very  complete  set  of  acts,  averaging  less  than  a  third  of 
the  acts  the  experimenters  thought  were  "worth  considering."  However, 
subjects  had  limited  time  to  respond,  and  their  performance  might  have 
been  better  had  they  been  allowed  unlimited  time. 

Decision  problem  structuring  is  the  process  of  explicitly  modeling  a 
decision  problem  for  purposes  of  Decision  Analysis.  It  is  curious  that 
considerable  effort  has  been  devoted  to  enhancing  act  generation 
performance  (and  other  predecision  processes)  given  the  paucity  of 
information  on  the  quality  of  unaided  hunan  problem  structuring. 
Nevertheless,  such  efforts  exist  (e.g.  Leal  and  Pearl,  1977;  Pearl,  Leal  & 
Saleh,  note  1;  Weiss  &  Kelly,  note  2;  Merkhofer,  Miller,  Robinson  & 
Korean,  note  3) .  11)6  work  of  Pearl  and  his  colleagues  is  of  particular 
interest  because  they  speculated  on  the  cognitive  mechanisms  involved  in 
act  generation. 


Given  the  near  absence  of  research  on  hunan  act  generation  and  the 
considerable  interest  in  improving  act  generation,  it  would  seem  desirable 
to  investigate  hunan  act  generation  capabilities.  The  present  study 
attempts  provide  a  preliminary  examination  of  act  generation,  one  that 
will  establish  the  basis  for  a  study  of  the  cognitive  processes  involved. 
Two  major  questions  will  be  addressed: 

1.  Tq  what  extent  can  subjects  generate  a  set  of  acts? 

2.  Hhafc  Is  the  potential  cost  any  failures  t£  generate  important  acts? 

The  first  of  these  questions  is  important  because  the  Pitz,  et.  al.  study 
addressed  the  question  of  capability  only  incidentally  to  its  main 
purpose — comparing  various  act  elicitation  techniques.  They  used  no 
criterion  of  performance,  and  allowed  only  a  limited  amount  of  time  for 
each  problem.  Thus,  their  data  are  less  than  definitive.  More  basic 
information  on  act  generation  performance  is  needed.  Can  subjects  generate 
most  of  the  important  acts  to  solve  a  problem,  or  is  their  unaided 
performance  so  deficient  that  even  more  attention  should  be  given  to 
aiding  act  generation  for  important  decisions? 

The  importance  of  the  second  question  becomes  clear  when  it  is  realized 
that  it  is  unnecessary  (except  perhaps  from  a  Utopian,  normative  view)  for 
the  decision  maker  to  generate  all  important  acts.  It  is  necessary  for  at 
least  one  satisfactory  act  to  be  generated,  and  it  may  be  sufficient  if 
only  a  few  very  attractive  acts  are  on  the  "menu".  After  fulfilling  the 
above  requirements,  the  practical  importance  of  failing  to  generate  all 
acts  may  be  small.  Practical  decision  makers  may  "satisfice",  choosing  the 
first  act  meeting  some  minimum  criterion  of  acceptability  (Simon,  1956) . 
Even  if  they  "optimize"  by  generating  a  number  of  acts  then  picking  the 
best,  the  difference  between  their  best  act  and  the  theoretically  optimum 
act  may  be  small.  Therefore,  a  second  interesting  question  regarding  act 
generation  performance  is  the  cost  of  failing  to  generate  the  optimum  act, 
or  the  "opportunity  loss"  (Raiffa,  1968)  due  to  deficiencies  in  act 
generation.  Even  though  subjects  fail  to  generate  the  "optimal"  act,  they 
may  generate  another  act  having  a  similar  utility.  In  this  case,  failure 
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to  generate  a  complete  menu  of  acts  would  be  of  little  importance.  If, 
however,  the  difference  in  utility  between  the  best  act  generated  and  the 
optimal  act  is  large,  then  there  would  be  cause  for  concern. 

Question  1  is  addressed  by  an  experiment  in  which  subjects  were  given  a 
realistic  decision  problem  and  were  asked  to  generate  any  act  that  might 
conceivably  solve  the  problem.  The  second  question  is  addressed  in  a 
second  section  of  the  paper  by  a  series  of  two  experiments.  In  the  first 
experiment  of  the  series,  acts  generated  by  previous  subjects  were 
screened  for  feasibility.  The  final  experiment  obtained  utility  estimates 
for  the  feasible  acts.  These  estimates  permitted  calculation  of 
"opportunity  loss". 

A  criterion  lor  act.  generation  performance 

The  seemingly  intractable  problem  in  a  study  of  act  generation  capability 
is  the  difficulty  in  creating  a  criterion  of  performance.  TVo  major 
approaches  can  be  used  to  create  such  a  criterion.  First,  one  can  refer  to 
authoritative  sources  (Fischhoff,  Slovic,  and  Lichtenstein,  1978)  such  as 
an  "expert's"  structure  of  the  act  generation  problem.  While  this  may  be 
an  excellent  approach  when  it  can  be  utilized,  no  such  authoritative 
sources  exist  for  most  interesting  decision  problems. 

A  second  approach  does  not  rely  on  expert  judgment,  but  instead  uses 
ordinary  subjects'  responses.  In  this  procedure,  described  in  more  detail 
in  Gettys,  Mehle,  Baca,  Fisher,  and  Manning  (note  4) ,  the  responses  of  a 
group  of  subjects  are  pooled  to  create  a  lower-bound  estimate  of  the 
number  of  possible  acts.  A  group  menu  of  acts  which  might  successfully 
solve  the  problem  is  created  by  taking  the  union  of  the  acts  suggested  by 
each  individual.  The  result  is  a  lower-bound  estimate  in  that  a  few  more 
potential  acts  would  be  discovered  if  more  subjects  were  added  to  the 
group.  This  approach  has  the  advantages  that  it  can  be  applied  to  a  wide 
variety  of  situations,  and  it  is  a  conservative  criterion  of  performance 
because  it  is  a  lower-bound  estimate.  Its  major  disadvantage  is  that  it  is 
necessary  to  add  many  subjects  before  the  pool  of  acts  ceases  to  grew. 
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Experiment  1 


Method 

Criteria  for  selecting  problems,  in  choosing  act  generation  problems  we 
were  concerned  with  the  issue  of  the  possible  interaction  between 
expertise  and  the  performance  of  the  subjects.  Ideally,  expert  populations 
might  be  preferred  over  college  student  populations,  but  the  pooling  and 
other  procecUres  employed  required  large  numbers  of  subjects.  We  used  over 
500  subjects  in  these  experiments  and  related  projects,  and  would  have 
found  it  difficult  to  find  even  50  expert  subjects.  Our  strategy  was  to 
attempt  to  find  meaningful  decision  problems  for  our  subjects —  problems 
for  wnich  they  were  suitably  "expert"  so  our  results  could  be  generalized 
to  other  populations.  Belated  work  on  hypothesis  generation  has  suggested 
expert  subjects  and  college  students  have  similar  deficiences  in  accessing 
information  in  memory  (Gettys,  et.  al. ,  note  4;  Mehle,  note  5). 

The  problems  were  chosen  with  the  following  criteria  in  mind: 

1.  The  problems  should  be  ones  with  which  the  subjects  are  knowledgable. 

2.  The  problems  should  be  "open-ended"  in  that  possible  acts  should  not  be 

easily  enumerable,  or  generated  by  any  mechanical  rule. 

3.  The  problems  should  be  tractable  in  the  sense  that  the  number  of  acts 
generated  by  the  subjects  should  not  be  so  large  as  to  be  unmanagable. 

4.  The  problems  should  resemble  problems  encountered  in  everyday  life, 
rather  than  being  highly  artificial  problems  such  as  those  used  by  some 
researchers  in  problem  solving,  (e.g.  "Missionaries  and  Cannibals"  or 
"Towers  of  Hanoi") . 

Problems.  TWo  act  generation  problems  were  used  in  this  investigation. 
After  completing  data  collection  for  the  first  problem,  we  decided  the 
results  were  of  sufficient  interest  to  warrant  including  a  second  problem 
as  a  check  on  the  generality  of  the  results  of  the  first  problem.  The  two 
problems  used  were  well-suited  to  our  subject  population.  The  first 
problem,  called  "Parking",  asked  the  subjects  to  assure  they  were  members 
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of  a  hypothetical  student  comnittee  appointed  by  the  officials  of  the 
University  to  provide  student  input  on  the  University's  parking  problem. 
Like  most  major  universities,  the  University  of  Oklahana  has  a  perennial 
parking  problem.  This  problem  is  discussed  frequently  in  the  student 
newspaper.  Most  students  engage  in  frequent,  if  not  daily,  hunts  for  a 
parking  space.  The  subjects'  task  was  to  list  all  acts  which  could  be 
taken  by  the  University  to  reduce  the  severity  of  the  parking  problem.  The 
text  of  the  parking  problem  is  given  below. 


PARKING 


It  is  difficult  to  find  a  place  to  park  at  CXI.  Although 
approximately  21,000  students  are  enrolled  and  about  3500 
faculty  and  staff  are  employed,  about  half  that  many  parking 
spaces  are  available. 


There  are  6850  parking  places  available  on  campus  for 
faculty/staff,  ccnmuters,  housing.  Law,  and  OGCE.  The  Lloyd 
Noble  Center  provides  an  additional  3500  spaces,  yielding  a 
total  of  10,350.  One  space  for  every  two  people  doesn't  sound 
too  bad,  but  anyone  wno  has  tried  to  finer  a  parking  space  at 
9:00  a.m.  knows  this  is  a  real  problem. 


e  a  member  of  a  .student  organization  which,  is 


•rgani _ 

researching  this  problem  for  officials  of  the  University.  Your 
task  is  to  suggest  as  many  possible  solutions  to  the  ccc  dttee 
as  you  can.  These  solutions  need  not  be  ^perfect";  often  good 
solutions  are  derived  from  ideas  which  at  first  seem  silly.  The 


„ .  .  _  _  __  simply 

possible  solutions  which  might  be  effective. 

It.  is  important  for  you  to  enter  all  options  which  occur  to  you. 
This  is  similar  to  "thinking  out  loud."  We  do  not  want  you  to 
?r  your  options  and  only  enter  the  ones  which  you  think  are 
Lcularly  good.  Put  down  all  options  whi,ch  gecur  to  ^ou,  good 


ir  bad.  Ydbr-  score  in  this  experiment  is  determined  by  the 
number  of  options  you  generate,  not  how  good  they  are. 


The  second  problem  involved  finding  a  place  to  live  for  an  impecunious 
Canadian  friend  who  arrived  on  campus  in  such  a  state  of  financial 
disarray  that  he  lacked  sufficient  funds  to  pay  for  a  dormitory  room  or  an 
apartment.  The  text  of  the  problem  was  carefully  worded  so  conventional 
solutions,  such  as  getting  a  job  or  asking  his  parents  for  more  money, 
were  not  viable  options.  This  problem  is  called  the  "Living"  problem,  and 
its  complete  text  is  found  in  Appendix  A. 

aifajfiCtfi*  Subjects  were  60  introductory  psychology  students  from  the 
University  of  Oklahoma  who  participated  in  the  experiment  for  course 
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credit,  Thirty  subjects  generated  acts  for  the  Parking  problem  and  30 
subjects  generated  acts  for  the  Living  problem.  Subjects  were  recruited 
who  knew  how  to  type,  and  had  to  demonstrate  they  could  type  at  least  20 
words  per  minute  to  participate  in  the  experiment. 

Instructions.  Hie  instructions  were  presented  by  a  computer .  Several  major 
topics  were  covered  in  the  instructions.  After  the  subject  was  told  the 
purpose  of  the  experiment  in  general  terms,  the  importance  of  responding 
with  all  possible  acts  was  emphasized  in  several  different  paragraphs. 
Subjects  were  instructed:  "...  We  are  interested  in  how  many  alternate 
solutions  you  can  dream  up.  It  is  not  important  that  these  solutions  be 
the  best  solutions  to  the  problem;  we  are  interested  in  even  remotely 
possible  solutions  that  occur  to  you."  In  another  section  of  the 
instructions  a  paragraph  was  devoted  to  exhaustively  searching  memory: 
"...Don't  give  up  on  a  problem  until  you  are  sure  you  have  run  out  of 
possible  solutions.  One  way  to  avoid  giving  up  too  soon  is  to  continue  to 
search  your  memory  for  a  while  even  though  you  believe  you  have  thought  of 
everything.  Usually,  a  number  of  other  possibilities  will  occur  to  you..." 
This  advice  was  reiterated  in  condensed  form  when  the  subjects  indicated 
they  had  completed  the  problem. 

Otner  parts  of  the  instructions  were  devoted  to  interacting  with  the 
computer's  menu  to  1)  Review  the  instructions  of  the  experiment,  2) 
Display  the  problem  on  the  screen  again,  3)  Enter  a  new  act,  4)  Review 
acts  already  generated  (to  avoid  duplication) ,  5)  Get  help  on  any  aspect 
of  the  menu,  and  6)  Terminate  the  experiment. 

After  the  subjects  had  read  these  instructions,  they  were  given  a  practice 
problem.  The  experimenter  helped  them  during  the  practice  problem  and  did 
not  let  the  subject  move  on  to  the  main  problem  until  it  was  clear  the 
subject  understood  the  experiment.  The  practice  problem  consisted  of 
generating  acts  which  might  be  taken  if  one  ran  out  of  gas  on  the  freeway 
and  had  no  money. 

After  completing  the  practice  problem,  subjects  were  shown  exemplar  acts 
generated  by  a  highly  creative  subject  for  the  sane  problem.  This  was  done 
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to  encourage  then  to  think  of  as  many  acts  as  possible. 

Subjects  were  told  there  was  only  one  problem  after  the  practice  problem, 
and  they  were  to  do  the  best  job  they  possibly  could  on  the  single 
problem.  They  were  also  told  they  could  take  as  much  time  as  they  needed 
to  complete  the  main  problem.  These  instructions  were  developed  on  the 
basis  of  several  pilot  studies,  in  which  it  was  found  working  on  a  single 
problem  leads  to  a  considerable  increase  in  the  number  of  responses 
generated. 

Procedure.  The  subjects  interacted  with  the  computer  by  reading  the 
textual  material  from  a  CRT  display  and  entering  their  acts  via  the 
keyboard.  The  previously-described  menu  of  casnands  was  used  by  subjects 
to  control  the  computer.  For  example,  subjects  generated  acts  by  first, 
hitting  the  "E"  key  which  signified  they  wanted  to  "Enter"  an  act,  then 
typing  their  act.  Subjects  were  then  asked  to  type  a  short  justification 
for  the  act.  This  was  done  to  aid  the  experimenters  in  interpreting 
subjects'  reasoning  behind  suggesting  sane  of  the  acts.  After  the 
justification  was  entered,  the  computer  returned  to  the  menu.  When 
subjects  chose  to  end  the  problem,  they  were  asked  to  think  again  of  ary 
acts  which  might  successfully  solve  the  problem. 

Subjects  then  generated  as  many  acts  as  they  could  for  either  the  Parking 
or  the  Living  problem.  The  experimenter  was  present  for  the  entire  session 
and  answered  questions  at  ary  time. 

Results  sod  Discussion 

Creating  the  act  sod  by  combining  equivalent  acts.  Subjects  generated  a 
total  of  335  acts  for  the  Parking  problem  and  362  acts  for  the  Living 
problem.  Mary  acts  were  equivalent  or  highly  similar,  althouc^i  the  words 
used  to  describe  the  acts  differed.  To  combine  equivalent  acts,  three 
experimenters  served  as  judges  to  independently  group  acts  which  they  felt 
were  equivalent.  For  example,  a  suggestion  to  build  a  six-story  parking 
garage  was  considered  to  be  equivalent  to  building  a  ten-story  garage.  The 
equivalence  judgnents  were  then  analyzed  by  a  computer  program.  Acts  which 
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were  classified  as  equivalent  by  all  three  independent  judges  were 
combined  into  one  act.  This  procedure  resulted  in  an  act  pool  of  128 
"unique"  acts  for  the  Parking  problem  and  155  "unique"  acts  for  the  Living 
problem. 

&  hierarchical  organization  of  the  acts.  It  is  not  reasonable  to  expect  a 
subject  to  generate  as  many  acts  as  the  pooled  responses  of  30  subjects.  A 
more  realistic  expectation  is  that  a  good  act  generator  should  be  able  to 
retrieve  examples  of  the  major  types  of  acts  from  memory.  Performance  can 
be  evaluated  by  examining  the  responses  of  an  individual  with  respect  to 
some  hierarchical  structure  created  by  logical  analysis  of  the  ideas 
embodied  in  the  unique  acts.  When  the  unique  acts  were  inspected  it  was 
clear  they  could  be  organized  into  a  hierarchical  structure.  Certain 
generic  ideas  were  shared  by  many  of  the  acts,  and  there  were  major 
variations  of  these  ideas,  as  well  as  minor  variations.  Because  the 
hierarchical  structure  is  tree- like,  we  named  the  generic  ideas  "limbs", 
major  variations  "branches",  and  minor  variations  "twigs".  By  comparing 
the  act  tree  generated  by  each  subject  with  the  "group"  tree  created  by 
pooling  the  responses  of  all  30  subjects,  we  can  identify  an  individual's 
omission  of  important  limbs  and  branches. 

Two  experimenters  structured  the  unique  acts  generated  for  the  Parking  and 
Living  problems.  The  results  are  shewn  in  Appendices  B  and  C, 
respectively.  The  limbs,  branches,  and  twigs  in  the  trees  are  denoted  by  a 
decimal  notation.  For  example,  the  notation  2.3.1  for  the  Parking  problem 
refers  to  a  proposal  to  build  a  belcw-ground  parking  stucture.  This  act  is 
a  member  of  generic  limb  2,  "Increase  available  space  for  parking",  and  is 
found  on  branch  3,  "Build  parking  structures",  a  branch  it  shares  with 
another  act,  "A  high-rise  parking  structure"  (2.3.2).  Most  limbs  contained 
many  branches,  and  some  branches  had  a  large  number  of  twigs.  Each  twig 
represents  at  least  one  act,  although  some  twigs  combine  several  acts.  The 
trees  were  created  with  a  much  more  liberal  definition  of  equivalence  than 
used  in  the  "three  judge"  procedure  to  keep  the  size  of  the  tree  within 
manageable  preportions.  Less  specific  acts  were  associated  with  a  branch 
or  even  a  major  limb,  rather  than  a  twig. 
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An  inspection  ot  Appendices  B  and  C  reveals  the  incredible  richness  of  the 
Parking  and  Living  trees  created  from  the  pooled  acts.  This  is  even  more 
remarkable  when  it  is  considered  the  trees  provide  lower-bound  estimates 
of  the  "hypothetical”,  asymptotic  trees  created  by  including  many  more 
subjects  in  the  pooling  process.  The  Parking  problem  has  seven  major 
limbs,  and  the  Living  problem  has  nine.  Examples  of  major  limbs  in  the 
Parking  problem  include  "Use  available  parking  space  more  efficiently", 
"Increase  available  space  for  parking",  "Reduce  demand  for  parking",  and 
"Change  parking  priorities".  Other  limbs,  such  as  "Provide  faculty  and 
Staff  with  housing  within  walking  distance"  and  "Frivolous  and  other 
responses”,  while  consisting  mostly  of  acts  that  are  patently  unrealistic, 
are  included  because  of  their  intrinsic  interest.  These  limbs  contain 
material  suggestive  of  some  strategies  subjects  use  during  act  retrieval. 
Limb  7,  "Responses  that  combine  two  or  more  options  from  above",  is  based 
on  the  response  of  one  subject  and  may  have  resulted  from  a 
misunderstanding  on  the  part  of  that  subject.  The  instructions  specified 
that  the  task  was  to  create  a  menu  of  possible  acts,  and  we  did  not 
specify  that  only  fine  act  would  be  picked  from  that  menu.  However,  it  is 
interesting  to  note  only  one  subject  specified  the  possibility  of 
combining  acts. 

The  tree  created  for  the  Living  problem  is  similar  to  the  Parking  problem 
tree  in  many  ways.  Examples  of  major  limbs  included  in  the  tree  for  the 
Living  problem  are  "Live  somewhere  without  paying  rent",  "Exchange  goods 
or  services  for  money  or  place  to  live",  "Obtain  money  through  other 
sources",  "Ask  someone  for  help",  "Try  to  change  regulations",  "Change 
current  plans",  and  "Prepare  in  advance".  Subjects  often  suggested  acts 
involving  changing  regulations  or  violating  the  constraints  placed  on  the 
problem  by  the  experimenter.  Again,  subjects  suggested  acts  having  a  small 
probability  of  successfully  solving  the  problem.  Those  acts  were  included 
in  two  categories,  "Long  shots”,  and  "Acts  which  will  not  solve  problem". 

fvy>a*r1irfHTYiT  A  tree  with  cluster  analysis,  it  could  be  argued  that  the 
structures  for  both  the  Parking  and  the  Living  trees  resulted  from  the 
idiosyncratic  thought  processes  of  the  individuals  who  created  them.  Other 
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individuals  might  have  produced  different  trees.  If  our  trees  are  to  be 
useful  in  assessing  the  completeness  of  act  generation  performance,  same 
indication  of  their  generality  should  be  available.  Manning  (note  6)  has 
provided  one  such  check  on  the  generality  of  the  tree  used  to  assess  the 
Parking  problem.  In  an  investigation  of  the  subjective  dimensions  used  by 
subjects  in  thinking  about  acts  for  the  Parking  Problem,  she  performed  a 
complete-linkage  cluster  analysis  on  acts  having  positive  utility.  Hie 
output  of  this  analysis  is  a  hierarchical  tree,  which  is  given  as  Appendix 
D.  Her  tree  includes  only  acts  generated  by  the  subjects  from  the  Parking 
problem  judged  to  be  of  positive  utility.  Despite  this  difference,  there 
are  real  similarities  between  the  experimenter-generated  tree  in  Appendix 
B  and  the  tree  resulting  from  cluster  analysis  in  Appendix  d.  Her  major 
limbs  1.,  "Do  more  with  University-owned  space"  and  2.,  "Expand  parking  to 
surrounding  areas"  are  similar  to  the  limb  2.,  "Increase  available  space 
for  parking"  in  Appendix  B.  Similarly,  her  limb  3.,  "Use  alternate  forms 
of  transportation"  is  similar  to  branch  3.3  in  Appendix  B,  "Introduce  or 
encourage  alternate  forms  of  transportation".  There  are  differences 
between  the  two  trees,  but  the  differences  are  not  great  enough,  in  our 
opinion,  to  render  the  experimenter-generated  trees  unusable  for  making 
judgments  about  the  completeness  of  human  act  generation. 

Individual  act  generation  performance,  individuals  in  the  Parking  and 
Living  problems  generated  an  average  of  11.2  and  12.1  acts,  respectively. 
Hie  number  of  acts  generated  for  the  Parking  and  Living  problems  ranged 
from  2  to  35.  Similar  large  individual  differences  in  performance  were 
observed  by  Manning,  et.  al.  (1980)  in  hypothesis  generation.  Subjects 
generated  an  average  of  3.3  limbs  for  the  Parking  problem  and  4.2  limbs 
for  the  Living  problem.  Subjects  generated  an  average  of  1.92  branches  for 
each  limb  they  generated  for  the  Parking  problem,  and  1.52  branches  for 
each  limb  generated  for  the  Living  problem. 

Hie  number  of  acts  generated  in  each  limb  or  branch  is  shewn  to  the  left 
of  each  limb  or  branch  in  the  trees  given  in  Appendices  B,  C,  and  D.  Hie 
frequency  of  acts  in  Appendix  D  is  lewer  because  only  positive  utility 
acts  were  included.  Also  included  is  the  probability  of  generating  at 


least  one  act  In  a  limb  or  branch.  For  example,  the  notation  "14”  followed 
by  "(.37)"  for  limb  1  of  the  Parking  problem  means  that  there  were  14  acts 
in  limb  1  and  11  of  the  30  subjects  (37%)  generated  at  least  one  of  the  14 
acts. 

Inspection  of  these  frequencies  for  the  Parking  problem  tree  yields 
several  conclusions.  First,  while  limb  2,  "Increase  available  space  for 
parking",  and  limb  3,  "Reduce  demand  for  parking",  were  generated  by 
almost  all  subjects  (f^.97  in  each  case) ,  other  major  limbs  such  as  limb  1 
and  limb  4  were  only  generated  by  about  a  third  of  the  subjects.  Limb  1, 
"Use  available  parking  space  more  efficiently",  is  an  important  limb  the 
University  is  currently  implementing.  Parking  lots  are  being  redesigned  to 
be  more  efficient  using  the  1.1,  1.2,  1.5,  1.6,  and  1.7  branches  shown  on 
the  tree.  Similarly,  rethinking  parking  priorities  (limb  4)  could 
noticeably  improve  the  parking  situation.  However,  two  out  of  three 
subjects  did  not  think  of  these  (obvious  in  hindsight)  limbs. 

Secondly,  while  most  subjects  generated  at  least  one  act  in  limbs  2  and  3, 
many  other  major  branches  which  one  would  expect  a  large  proportion  of  the 
subjects  to  generate  did  not  show  large  proportions.  For  example, 
providing  incentives  such  as  reserving  parking  places  for  people  who 
car  pool  (3.4.2)  would  be  an  effective  act  which  would  cost  the  University 
almost  nothing,  yet  this  act  was  only  rarely  suggested  by  subjects. 

A  similar  analysis  can  be  made  using  the  tree  constructed  from  cluster 
analysis  for  the  Parking  problem.  Only  30%  of  the  subjects  thought  of 
branch  1.2,  "Reorganize  current  parking  areas",  yet  this  is  the  most 
cost-effective  way  to  increase  the  available  parking  given  the  strong 
shift  to  subccmpact  cars  in  the  university  community.  Only  37%  of  the 
subjects  thought  of  branch  3.3,  "Promote  other  forms  of  transportation". 
Bicycle  riding  would  probably  become  much  more  popular  if  a  way  could  be 
found  to  reduce  bicycle  thefts  and  bicycle  paths  were  built  to  separate 
bicyclists  from  pedestrians  and  motorists. 

In  the  Living  problem,  90%  of  the  subjects  generated  at  least  one  act  in 
limb  1.0,  "Live  with  someone  without  paying  rent”,  and  in  limb  3.0, 
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"Obtain  money  from  other  sources".  Eighty  percent  of  the  subjects 
generated  at  least  one  act  in  limb  2.0,  "Exchange  goods  or  services  for 
money  or  a  place  to  live".  However,  only  27%  of  the  subjects  thought  to 
ask  someone  else  for  housing  suggestions  (limb  4)  and  only  60%  considered 
having  the  friend  modify  his  plans  (limb  6) .  While  most  subjects  generated 
at  least  one  act  in  limb  1,  "Live  with  someone  without  paying  rent",  the 
majority  only  considered  having  the  friend  live  with  someone  he  knows 
rather  than  explore  other  sources.  In  limb  2,  "Exchange  goods  or  services 
for  money  or  a  place  to  live",  63%  of  the  subjects  thought  of  selling 
goods  or  services  to  make  money,  but  less  than  half  considered  exchanging 
his  services  for  living  quarters.  In  limb  3,  "Obtain  money  from  other 
sources",  only  a  third  of  the  subjects  suggested  applying  for  financial 
assistance. 

Obviously  there  is  a  subjective  element  in  these  analyses.  Readers  will 
have  to  inspect  the  trees  and  reach  their  own  conclusions.  We  were 
surprised  by  the  deficiencies  in  subjects'  performance  even  though  our 
earlier  research  on  hypothesis  generation  suggested  subjects  would  fail  to 
think  of  important  acts. 

The  source  of  these  failures  is  an  interesting  topic  for  speculation.  The 
problems  were  picked  so  we  could  argue  that  most  important  acts  would  be 
"available"  in  memory,  but  not  neccessarily  "accessible"  (Tulving  and 
Pearl stone,  1966).  In  other  words,  we  believe  that  the  requisite 
information  is  available  in  the  memories  of  most  of  our  subjects  die  to 
their  experiences  with  personal  transportation,  or  their  experiences  with 
finding  acccnmodations  for  themselves  or  their  friends.  Therefore,  we  are 
inclined  to  attribute  most  of  the  difficulty  in  act  retrieval  to  failure 
to  access  information  available  in  memory.  Perhaps  an  expert  would  have 
less  difficulty  accessing  material  used  on  a  daily  basis.  On  the  other 
hand,  we  believe  that  important  decisions  are  not  necessarily  made  by 
experts  who  make  similar  decisions  on  a  daily  basis.  Often  "generalists" 
such  as  politicians,  military  officers,  and  lawyers  are  found  in  decision 
making  roles.  Do  they  have  difficulties  in  accessing  information  similar 
to  our  subjects'  difficulties?  We  suspect  so. 
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We  do  not  believe  many  of  the  failures  to  generate  important  limbs  and 
branches  are  due  to  a  hyper-critical  attitude  on  the  part  of  our  subjects 
towards  the  acts  they  retrieved  from  memory.  As  can  be  seen  from  an 
inspection  of  the  trees,  many  responses  are  counter-productive  or  useless, 
suggesting  the  subjects  were  trying  to  follow  instructions  by  responding 
with  everything  that  came  to  mind.  Why  would  they  make  frivolous  responses 
rather  than  suggest  acts  which  might  have  been  effective? 

We  cannot  preclude  a  motivational  explanation  for  some  failures  to  think 
of  important  limbs  and  branches.  We  went  to  considerable  pains  to  design 
instructions  and  procedures  for  our  experiments  to  motivate  the  subjects. 
Most  subjects  spent  about  an  hour  in  the  experiment,  which  was  restricted 
to  a  single  problem.  Many  subjects  found  the  experiment  intrinsically 
interesting  and  treated  it  as  a  contest  or  a  game.  Nevertheless,  seme 
retrieval  failures  probably  were  due  to  insufficient  motivation. 

Our  conclusion  from  inspecting  these  data  is  that  the  tree  produced  by  the 
individual  subject  often  lacks  important  limbs  and  branches.  We  presented 
almost  all  the  data  used  in  reaching  this  conclusion  so  the  readers  can 
reach  their  own  conclusions.  If  our  conclusion  is  accepted,  the  next 
logical  step  is  to  assess  the  cost  of  failing  to  generate  a  complete  tree. 


Fbrpe^i  merit  £ 

The  second  major  question  of  opportisiity  loss  can  be  addressed  by  finding 
the  utility  of  all  unique  acts  in  the  "pooled-acts"  tree.  Once  the 
utilities  of  various  branches,  limbs,  and  twigs  are  known,  the  tree 
produced  by  each  subject  can  be  compared  to  the  pooled-acts  tree,  and 
opportunity  loss  can  be  calculated.  If  subjects  do  not  generate  a  complete 
tree,  but  generate  all  important  limbs  and  branches,  opportunity  loss  will 
be  negligible.  If,  however,  important  limbs  and  branches  are  omitted, 
opportunity  loss  will  be  large. 

Figuratively,  the  first  experiment  examined  how  well  subjects  can  sketch 
the  pooled  acts  tree,  and  the  remaining  experiments  examined  whether  the 


brandies  they  sketch  out  are  "heavy  with  fruit".  An  examination  of  the 
completeness  of  the  structure  is  incomplete  without  an  assessment  of  the 
consequences  of  failing  to  generate  branches  and  limbs.  Whereas  the 
previous  analysis  relied  heavily  on  such  words  as  "important"  or  "useful" 
and  invited  the  reader  to  agree  or  disagree  with  this  judgment,  an 
opportunity  loss  calculation  injects  an  element  of  objectivity  into  the 
analysis. 

Our  primary  difficulty  in  performing  an  opportunity  loss  calculation  was 
that  the  "pooled-act”  trees  were  much  larger  than  we  anticipated.  We  had 
planned  to  use  a  version  of  Multi-Attribute  Utility  scaling  to  estimate 
the  utility  of  each  act,  but  the  large  number  of  unique  acts  made  this 
impossible.  Instead,  we  were  forced  to  settle  for  first,  screening  the 
unique  acts  to  identify  feasible  acts  of  positive  utility,  and  second, 
evaluating  the  utility  of  the  screened  acts  by  using  a  global  utility 
estimation  technique. 

Consequently,  we  performed  experiment  2  to  provide  a  preliminary  screening 
of  the  acts.  The  technique  used  was  to  ask  subjects  to  identify  acts 
having  positive  utility.  Experiment  3  provided  global  utility  estimates 
for  acts  judged  to  be  of  positive  utility  by  most  of  the  subjects  in  the 
preliminary  screening. 

Method 

Subjects.  Subjects  were  60  introductory  psychology  students  from  the 
University  of  Oklahoma  who  participated  in  the  experiment  for  course 
credit.  Thirty  subjects  saw  the  Parking  problem  and  30  subjects  saw  the 
Living  problem. 

Pranodm-A.  The  experiment  was  computer-controlled.  Subjects  first  read  the 
practice  problem  seen  by  subjects  in  the  act  generation  experiment.  The 
practice  problem  dealt  with  acts  to  be  taken  if  the  subject  ran  out  of  gas 
and  had  no  money.  Subjects  were  shown  4  sample  acts  and  were  asked  to 
respond  "yes"  or  "no"  to  the  question  of  whether  taking  each  specified  act 
would  leave  them  better  off.  A  response  to  the  question  was  made  by  typing 
either  "Y"  or  "N”  on  the  keyboard.  After  each  response,  subjects  were 
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given  an  opportunity  to  look  at  the  problem  again  or  to  move  on  to  the 
next  act. 

Following  the  practice  problem,  subjects  read  either  the  Parking  or  the 
Living  problem.  Subjects  were  told  their  task  was  to  decide  whether  taking 
the  specified  acts  would  leave  the  involved  parties  better  off  than  they 
currently  were.  Subjects  were  also  encouraged  to  weigh  the  costs  of  taking 
any  act  against  its  potential  benefits  when  deciding  whether  the  act  good. 
Subjects  again  evaluated  acts  by  typing  either  "Y"  or  "N"  on  the  keyboard. 
Subjects  who  saw  the  Parking  problem  evaluated  128  acts  and  subjects  who 
saw  the  Living  problem  evaluated  155  acts. 

n^miit-g  and  ni  going -inn.  The  number  of  subjects  who  thought  an  act  was  of 
positive  utility  was  determined.  If  the  majority  of  subjects  in  the 
preliminary  screening  experiment  rated  an  act  as  having  positive  utility, 
it  was  retained  for  use  in  the  utility  experiment.  Fifty-seven  acts  were 
classified  as  being  of  positive  utility  in  the  Parking  problem,  and  67  in 
the  Living  problem. 

The  procedure  used  for  preliminary  screening  was  somewhat  crude,  but  it 
should  be  remembered  that  opportunity  loss  is  calculated  by  finding  the 
difference  between  the  best  act  on  the  "pooled  acts”  tree  and  the  best  act 
on  a  subject's  tree.  Consequently,  the  utility  of  acts  having  negative 
utility  need  not  be  known,  as  the  influence  of  negative  utility  acts  on 
the  opportunity  loss  calculation  is  negligible  (except  in  pathological 
cases) . 


Experiment  2 

Tfrfihnlaiftfl  iQL  aafffiMinq  generation  performance 

• 

Many  of  the  acts  in  the  Parking  and  Living  problems  are  not 
mutually-exclusive.  Thus,  it  should  be  possible  to  create  a  "portfolio* 
(Cozzolino,  1974)  of  acts  to  address  the  problem.  For  example,  parking  at 
the  University  of  Oklahoma  could  be  improved  by  redesigning  existing 
parking  lots,  building  new  lots,  improving  mass  transit,  etc.  On  the  other 
hand,  the  Canadian  student  can  only  live  in  one  place  at  a  time,  2d  though 


other  acts,  such  as  getting  more  money,  could  be  combined  with  acts 
suggesting  where  he  might  live.  Therefore,  a  subject's  performance  should 
be  evaluated  in  terms  of  both  the  best  act  generated  and  the  number  of 
high  utility  acts  generated,  because  many  acts  could  be  taken 
simultaneously . 

Portfolio  analysis  is  a  formal  technique  to  identify  the  best  mixture  of 
acts  to  be  taken.  Unfortunately,  the  problems  we  are  considering  are  not 
suitable  candidates  for  such  an  analysis  because  some  acts  are  mutually 
exclusive.  Instead,  we  developed  an  analysis  which  embodies  some  of  the 
ideas  of  portfolio  analysis  and  addresses  other  dimensions  of  performance 
as  well. 

Suppose  a  subject  produces  an  act  tree  consisting  of  limbs,  branches,  and 
twigs.  Each  act  twig  has  an  associated  utility  value  obtained  from  a 
utility  scaling  procedure.  Also  imagine  a  comparable  tree  created  by 
pooling  the  acts  of  the  entire  group  of  30  subjects.  The  "group"  tree  will 
have  many  more  branches  and  twigs,  but  if  subjects  perform  well  their 
"individual"  trees  should  resemble  a  pruned  "group"  tree  containing  only 
the  best  limbs,  branches  and  twigs. 

A  tree  can  be  pruned  at  various  levels.  Twigs  can  be  pruned  from  a  branch 
until  only  the  highest-utility  twig  remains.  After  twigs  are  thus  pruned 
from  each  branch,  branches  retaining  the  best  twigs  can  be  pruned  from  a 
limb  until  only  the  highest-utility  branch  remains.  Finally,  limbs 
retaining  oily  their  best  branch  and  twig  can  be  pruned  until  only  the 
highest-utility  limb  remains. 

If  both  the  "group"  and  "individual"  trees  were  pruned  to  their  single 
best  limb,  branch  and  twig,  the  act  having  maximum  utility  for  each  tree 
would  be  identified.  The  difference  between  the  maximum  utility  act  for 
the  "group"  tree  and  an  "individual"  tree  would  be  the  "opportunity  loss" 
resulting  from  the  subject's  failure  to  generate  the  best  possible  act. 

If,  however,  for  both  trees  pruning  stops  after  each  branch  is  pnned  so 
only  the  best  twig  (in  a  maximum  utility  sense)  remains,  then  a  pruned 
tree  would  consist  of  limbs  and  branches  containing  only  the  best  twig 


available  for  that  limb  and  branch.  If  the  analysis  is  extended  to  these 
additional  limbs  and  branches,  an  idea  of  the  variety  and  completeness  of 
the  subject's  tree  structure  can  be  obtained.  A  comparisicn  between  the 
best  limb  and  branch  of  the  two  can  still  be  made  to  calculate  opportunity 
loss.  The  additional  limbs  and  branches  can  be  examined  to  assess  the 
completeness  of  the  "individual"  tree.  We  have  named  this  analysis  "limbs 
and  branches". 

Once  the  tree  has  been  prtned  to  this  extent,  it  is  a  simple  matter  to 
prune  it  further,  so  each  limb  retains  only  its  maximum  utility  branch  and 
twig.  The  analysis  performed  on  this  type  of  tree  is  named  "limbs  only”. 
The  "limbs  and  branches"  analysis  and  the  "limbs  only"  analysis  provide 
somevnat  complementary  information.  If  a  "limbs  and  branches”  analysis  is 
performed,  a  ccmparision  of  the  group  tree  with  the  individual  tree  is 
essentially  an  examination  of  how  well  a  subject  generates  limbs  and 
branches.  If,  on  the  other  hand,  a  "limbs  only  analysis  is  performed,  the 
empnasis  is  on  how  well  a  subject  generates  various  limbs. 

Stppose  that  the  various  versions  of  the  group  and  individual  trees 
outlined  above  are  created.  One  way  to  simmarize  subjects'  for  limbs  only 
performance  is  to  create  a  cumulative  utility  function  which  sums  the 
utilities  of  the  limbs  in  order  of  decreasing  utility.  This  is  done  by 
first  taking  the  highest  utility  limb,  adding  this  utility  to  the 
second-highest  utility  limb,  etc.  A  similar  analysis  may  be  performed  on 
the  trees  consisting  of  limbs  and  branches.  The  cumulative  utility 
function  resulting  from  this  process  increases  monotonically,  and 
eventually  is  aysmptotic.  If  the  cumulative  utility  functions  are 
calculated  and  plotted  for  the  appropriate  versions  of  the  group  and 
individual  trees,  a  sunmary  measure  is  created  which  captures  both  a 
subject' 8  ability  to  generate  the  "best"  act,  and  the  ability  to  generate 
a  variety  of  other  good  acts.  Hie  difference  between  the  first  points  on 
two  of  these  functions  is  "opportunity  loss",  and  the  differences  between 
their  remaining  points  can  be  used  to  assess  the  "oonpleteness"  of 
performance. 
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utility  PHtimatinn  technique.  The  utility  estimation  technique  employed 
was  direct  utility  estimation.  Subjects  were  given  two  "anchor"  acts,  one 
identified  as  having  a  value  of  0,  the  other  a  value  of  100.  They  then 
made  global  utility  estimates  for  all  other  acts  by  canparing  each  act 
with  the  two  anchor  acts.  If  subjects  believed  an  act  had  a  higher  utility 
thcin  the  act  identified  as  having  a  utility  of  100,  they  were  free  to 

assign  it  a  larger  number.  If  they  believed  an  act  had  a  lower  utility 

than  the  0  utility  anchor,  they  could  assign  it  a  negative  number.  The 
acts  which  served  as  anchors  were  selected  using  the  results  of  experiment 
2.  For  the  Parking  problem,  the  act  "Improve  the  trolley  and  Campus  Area 
Rapid  Transit  System..."  was  used  as  the  upper  anchor  and  was  assigned  a 
value  of  100  because  more  subjects  (29  out  of  30)  classified  the  act  as 
being  of  positive  utility  than  ary  other  act  in  experiment  2.  A  utility  of 
zero  was  assigned  to  the  act  of  doing  nothing  about  the  Parking  problem. 
The  Living  problem  differed  from  the  Parking  problem  in  that  the  act  "do 

nothing”  was  not  viable,  as  the  student  needed  a  place  to  live. 

Consequently,  we  picked  an  act  for  a  lower  anchor  which  few  subjects  (3 
out  of  30)  in  experiment  2  felt  was  of  positive  utility.  This  act  was  "... 
hide  in  the  library  and  sleep  there  at  night...".  While  some  students 
resort  to  similar  tactics,  this  is  obviously  a  poor  solution  to  the 
problem.  The  act  chosen  to  be  the  upper  anchor  involved  appealing  to 
relatives  for  help  and  was  picked  because  all  subjects  thought  it  was  of 
positive  utility. 

The  two  utility  scales  defined  for  the  Parking  and  the  Living  Problems  are 
not  commensurate,  because  they  have  different-valued  intervals. 
Consequently,  comparisicns  between  the  Parking  and  Living  problems  cannot 
be  made  directly,  because  the  scales  are  technically  interval  scales  and 
are  only  unique  up  to  a  linear  transformation.  This  issue  will  be 
discussed  in  more  detail  later. 

Subjects.  Twenty  subjects  were  used  for  each  problem.  They  were  recuited 
from  the  same  population  as  previous  subjects. 
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instructions.  The  instructions  to  the  subjects  were  similar  to  those  used 
in  experiment  2.  Subjects  were  told  to  rate  acts  generated  by  other 
subjects  by  making  numerical  utility  estimates.  They  were  instructed  that 
their  global  estimates  should  incorporate  the  costs  of  implementing  a 
particular  act  and  possible  outcomes  resulting  from  that  act. 

Procedure.  The  procedure  for  the  practice  problem  was  the  same  as  in 
experiment  2,  except  subjects  made  direct  utility  estimates.  The  computer 
screen  displayed  three  acts.  The  two  anchor  acts  were  always  present  at 
the  top  and  bottom  of  the  screen,  and  were  labeled  0  and  100.  The  act  to 
be  judged  was  displayed  in  the  middle  of  the  screen  between  the  two 
anchors.  Subjects  entered  a  utility  value  into  the  computer  terminal  which 
represented  their  evaluation  of  the  utility  of  the  act  to  be  rated  as 
compared  with  the  two  anchor  acts.  The  computer  then  advanced  to  the  next 
act  to  be  rated. 

When  the  subjects  finished  the  practice  problem,  they  read  the  text  of 
either  the  Parking  problem  or  the  Living  problem.  They  made  direct  utility 
estimates  for  57  acts  from  the  Parking  problem,  or  67  acts  from  the  Living 
problem.  To  insure  subjects  would  be  conscientious,  they  were  required  to 
justify  the  magnitude  of  their  response  for  six  randomly  chosen  acts.  The 
subjects  did  not  know  for  which  acts  they  would  be  required  to  justify 
utility  estimates  until  after  they  entered  the  estimate. 

fleauJLtfi  sod  discussion 

The  subjects'  utility  estimates  were  examined  by  plotting  histograms  of 
the  distribution  of  the  estimates  for  each  problem.  Almost  all  histograms 
were  unlmodal,  but  there  was  considerable  variation  among  the  subjects' 
estimates.  The  median  utility  estimate  was  used  as  a  measure  of  central 
tendency  to  reduce  the  influence  of  outlying  scores. 

The  "liirhe  only"  arri  "Hmha  and  branches"  analyses.  A  computer  program 
performed  the  "limbs  only"  and  the  "limbs  and  branches"  analyses  described 
above.  It  also  performed  a  similar  analysis  on  unpruned  trees,  but  the 
results  of  this  analysis  were  so  similar  to  those  of  the  "limbs  and 
branches"  analysis  that  they  will  not  be  reported.  The  input  to  the 
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program  was  the  classification  of  acts  created  by  the  experimenters  and 
the  median  utility  estimates  obtained  from  experiment  3.  The  output  of  the 
program  was  a  cumulative  utility  function  for  the  group  tree  consisting  of 
pooled  acts  generated  by  all  subjects.  This  function  shows  the  cumulative 
utility  for  the  total  number  of  limbs  for  the  "limbs  only"  analysis,  and 
the  ten  highest  utility  limbs  and  branches  for  the  "limbs  and  branches" 
analysis.  In  each  case  the  first  point  in  the  function  is  the  maximum 
utility  act,  the  second  point  is  the  second-ranked  act,  etc.  Table  1  shows 
the  data  used  in  the  group  "limbs  and  branches"  analysis  for  the  Parking 
problem.  The  frequencies  of  generating  acts  illustrate  individual 
performance  and  were  not  a  part  of  the  group  analysis. 


Table  1 

The  best  10  limbs  and  branches  from  the  "group"  Parking  problem  tree 


Utility 


Number  of 

subjects 

generating 


Action 


100 

77.5 
60 
60 

57.5 
55 
55 
51 
50 
50 


2 

19 

2 

1 

1 

3 

2 

3 

1 

3 


improve  the  trolley  and  CART  systems. 

Build  a  high-rise  parking  lot. 

Put  more  small  car  parking  spaces  in  student  lots. 
Have  employees  park  at  Lloyd  Noble,  use  trolley. 
Build  lots  around  Norman,  shuttle  in  students. 
Reduce  the  price  of  parking  stickers  for  car pools. 
Advertise  advantages  of  riding  bike,  motorcycle. 
Have  more  selection  of  afternoon,  evening  courses. 
Use  sane  OU  service  vehicle  parking  for  students. 
Use  extra  areas  around  fraternities  for  overflow. 


616  30  possible 
sum  of  for  each  act 
utilities 


The  acts  shown  in  table  1  are  the  10  best  acts  from  the  group  tree  for  the 
Parking  problem.  The  cumulative  utility  function  was  obtained  from  the 
column  of  utilities  shewn  in  the  table.  Also  shown  is  the  number  of 


subjects  generating  each  act.  The  individual  analyses  were  similar  to 
this,  with  one  important  exception.  The  ten  best  acts  for  an  individial 
were  not  necessarily  those  shown  in  table  1,  but  were  the  ten  best  acts 
generated  by  the  individual.  An  inspection  of  the  number  of  subjects 
generating  the  acts  used  in  the  group  analysis  shews  the  typical  subject's 
ten  best  acts  included  sane  acts  fran  the  group  analysis,  but  also 
included  many  lower-utility  acts  not  shewn  in  table  1.  Therefore,  the 
individual  analyses  were  similar  in  principle  to  the  group  analysis,  but 
were  based  on  individual  performance. 

The  results  of  the  "limbs  and  branches"  and  the  "limbs  only"  analyses  are 
shown  in  figures  1  through  4  for  the  Parking  and  Living  problems.  Each 
figure  shows  the  "group"  cumulative  utility  function,  the  "mean"  function 
obtained  by  averaging  the  cumulative  utility  functions  for  each  subject, 
and  the  "best"  function  obtained  from  the  best  subject  in  the  analysis. 
The  difference  beween  the  first  point  of  the  "group"  function  and  the 
first  point  of  either  of  the  other  functions  is  the  opportunity  loss  die 
to  failure  to  generate  the  highest  utility  act. 

Opportunity  loss  results.  The  opportunity  loss  due  to  failure  to  generate 
the  best  act  was  29  utiles  (100-71)  for  the  Parking  problem  and  18.15 
utiles  for  the  Living  problem  (100-81.85).  One  way  of  thinking  about  the 
meaning  of  the  opportunity  loss  figures  is  to  imagine  the  utility  distance 
between  the  average  subject's  performance  and  the  upper  and  lower  anchors 
used  in  experiment  2.  The  utility  of  the  best  act  for  the  average  subject 
in  the  Parking  problem  was  71.  This  utility  value  is  71  percent  of  the 
distance  fran  the  lower  anchor,  and  29  percent  of  the  distance  to  the 

upper  anchor.  The  utility  of  the  best  act  for  the  average  subject  in  the 
Living  problem  was  81.85.  This  utility  value  is  81.85  percent  of  the 

distance  fran  the  lower  anchor  and  18.15  percent  of  the  distance  to  the 
upper  anchor.  It  should  be  recalled  that  these  opportunity  loss  figures 
are  lover-botnd  estimates.  The  actual  opportunity  loss  values  may  be 

aomwnat  higher. 

One  camot  conclude,  however,  that  the  performance  of  subjects  in  the 

Living  problem  was  superior  to  the  performance  of  subjects  in  the  Parking 


SC 7  RANKING 


LEGEND-. 


TREE  tt-K-w  BEST  a-B-e  GROUP 


-*-*  MEAN 


FIGURE  1.  CUMULATIVE  UTILITY  FUNCTIONS  FOR 
"LIMBS  AND  BRANCHES"  ANALYSIS 


PARKING  PROBLEM 


65CM 


600-1 


550-1 


5004 


C  450- 
U 

M 

U  400- 

L 

fl 

7  350- 
I 

V 

E  300- 
U 

7  250- 
1 

L 

1  200- 

7 

T 

150- 


100-^ 


50-1 


0- 

L _ _ _ 

i  r 

0  1 

1  i - 1 - 1 

2  3  4  5 

1  1 

6  7 

RC7  RRNK  JNG 

LEGEND-.  WEE 

K-  K  X  BE57  0-0  -B  HERN 

66*  707 RL 

FIGURE  2.  CUMULATIVE  UTILITY  FUNCTIONS  FOR  "LIMBS  ONLY"  ANALYSIS 


RC7  flRNK JNC 


LEGEND 


FIGURE 

ff 


•  IRK  X-H-H  BEST  O  O  Q  CROUP  AAA  HERN 


3.  CUMULATIVE  UTILITY  FUNCTIONS  FOR 
LIMBS  AND  BRANCHES"  ANALYSIS 


LIVING  PROBLEM 


FIGURE  4.  CUMULATIVE  UTILITY  FUNCTIONS  FOR  "LIMBS  ONLY"  ANALYSIS 


problem  because  the  two  utility  scales  are  not  corrmensurate.  Utility 
scales  are  unique  up  to  a  linear  transformation.  No  data  are  available  to 
translate  the  utility  units  of  the  Parking  scale  into  the  units  of  the 
Living  scale.  Our  impression  is  that  the  opportunity  loss  values  are 
roughly  comparable  for  the  two  problems.  It  will  be  recalled  that  the  act 
serving  as  the  lower  anchor  in  the  Parking  problem  was  to  do  nothing, 
whereas  in  the  Living  problem  it  was  to  sleep  in  the  Library.  Therefore, 
the  anchors  in  the  Living  problem  may  encompass  a  greater  range  of  utility 
than  the  anchors  for  the  Parking  Problem,  suggesting  each  utility  point  in 
the  Living  problem  scale  has  greater  value  than  each  point  in  the  Parking 
problem  scale.  Therefore,  the  opportunity  loss  of  29  utiles  for  the 
Parking  problem  may  represent  about  the  seme  loss  as  the  18.15  utile  loss 
for  the  Living  problem. 

Qid  subjects  generate  many  high-valued  limbs?  The  question  of  the 
completeness  of  each  subject's  tree  is  pertinent  to  the  topic  of  decision 
problem  structuring.  Decision  analysis  employs  many  powerful  and 
sophisticated  techniques  to  decide  if  an  act  should  be  included  in  a 
decision  analysis.  Obviously,  if  a  decision  maker  fails  to  generate  the 
act,  the  benefits  of  subsequent  decision  analysis  are  lost.  Decision 
analysis  depends  on  the  quality  of  the  decision  maker's  work  when 
generating  elements  of  the  structure.  This  issue  can  be  addressed  by 
examining  the  remainder  of  the  cumulative  utility  functions  shown  in 
figures  1  through  4.  The  difference  in  utility  between  the  first  and  last 
points  on  each  cumulative  utility  function  is  related  to  the  completeness 
of  the  associated  tree. 

Table  2  displays  the  differences  between  the  "group”  and  "mean”  cumulative 
utility  functions.  For  example,  in  the  "limbs  and  branches”  analysis  for 
the  Parking  problem,  the  difference  between  the  first  and  last  points  of 
the  "mean"  cumulative  utility  function  was  97.6  (168.6-71).  The  difference 
between  the  first  and  last  points  of  the  "group”  cumulative  utility 
function  was  516  (616-100).  Another  colunn  shows  the  ratios  (expressed  as 
percents)  of  the  utility  differences  for  the  "group”  function  to  the 
utility  differences  for  the  "mean”  function  for  each  prohlero. 
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Table  2 

Utility  Differences  from  the  Cumulative  Utility  functions 


Parking  Problem 


Living  Problem 


"Mean" 

"Group" 

Percent 

"Mean"  "Group" 

Percent 

"limbs  and 

branches 

analysis" 

97.6 

516 

528% 

201.9 

750 

371% 

"limbs  only 
analysis*^ 

52.2 

187 

358% 

129 

463.5 

359% 

Table  2  reveals  an  interesting  result.  Subjects  are  unable  to  generate  a 
complete  assortment  of  hi#  utility  acts  such  as  those  a  Decision  Analyst 
might  hope  to  elicit  from  a  client.  Subjects  do  not  generate  many  hi# 
utility  limbs  and  branches.  When  the  analysis  is  confined  to  limbs,  a 
similar  picture  emerges.  In  fact,  after  inspecting  the  data  from  which  the 
opportunity  loss  calculations  were  derived,  we  have  developed  a  rather 
unflattering  picture  of  human  act  generation  capabilities. 

First,  many  acts  occurring  to  subjects  are  of  zero  or  negative  utility. 
Subjects  in  the  Parking  problem  generated  an  average  of  11.2  acts.  Of 
these  11.2  acts,  4.4  or  39%,  on  the  average,  were  judged  to  be  of  positive 
utility.  Only  a  few  of  the  4.4  positive  utility  acts  generated  by  the 
average  subject  were  of  hi#  utility.  The  results  from  the  Living  problem 
were  similar.  The  average  subject  generated  12.1  acts.  Of  these  12.1  acts, 
6.5,  or  54%,  cm  the  average  were  of  positive  utility.  Subjects  in  the 
Living  problem  tended  to  generate  more  high-utility  acts,  and  tended  to 
generate  acts  of  higher  utility.  Table  3  summarizes  the  data  on  which 
these  conclusions  were  based.  It  gives  the  mean  number  of  acts  in  each 
utility  class  interval  for  the  two  problems  for  indivdual  subjects.  Also 
given  are  the  number  of  acts  from  the  group  tree  in  each  utility  class 
interval. 


Table  3  presents  the  relationship  between  utility  of  acts  and  the 
frequency  with  which  the  acts  were  generated  by  individual  subjects.  A 
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"perfect"  act  generator  should  perform  at  least  its  well  as  the  group  tree, 
but  we  do  not  expect  our  subjects  to  approach  perfection.  The  cumulative 
utility  functions  address  subject  performance  in  a  more  realistic  way. 
Table  3  shows,  however,  subjects  generate  only  a  few  viable  acts,  far 
fewer  than  would  be  desirable  from  the  point  of  view  of  decision  analysis. 


Table  3 

The  Utility  Distribution  of  Acts  Generated  by  Subjects 
Compared  to  the  Group  Tree 


Utility  values 
(Midpoints  of 
interval) 


Parking  Problem 
individual  group 


6:5? 

■M 


** 


75 

5 

6 

11 


Living  Problem 
individual  group 
5.5i 


.61 

$ 

.07 


1 

0 

2 


:B!  7 

hi? 

:8 


4 

6 

12 

4 

3 

5 


*  This  class  interval  contains  acts  eliminated  in 
experiment  2.  Mary  of  these  acts  would  have  had 
negative  utility  estimates  had  they  been  included 
in  experiment  3. 

**  These  entries  are  the  mean  number  of  acts  having 
utilities  within  the  specified  range. 


A  check  q D  the  generality  q£  these  results  Xia  Cluster  analysis.  The 
"limbs"  and  "limbs  and  branches"  analyses  described  above  were  repeated 
for  the  tree  constructed  for  the  Parking  problem  using  cluster  analysis. 
Graphs  of  the  emulative  utility  functions  for  these  analyses  are  shown  in 
Appendices  E  and  F.  Similar  conclusions  were  reached  from  the  analyses. 
The  opportunity  loss  calculation  was  unchanged  in  this  analysis,  as  it  is 
not  affected  by  the  structure  of  the  tree  used  in  the  analysis.  Utility 
differences  were  calculated  using  the  tree  derived  from  the  cluster 
analysis  and  are  reported  with  the  corresponding  results  from  table  2  in 
parentheses  following  each  result.  For  the  "limbs  and  branches"  analysis, 
the  "group"  function  had  a  utility  difference  of  539  (516)  utiles.  The 
"mean"  individual  function  showed  a  utility  difference  of  119.5  (97.6) 
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utiles.  A  ratio  of  451%  (528%)  was  found  between  the  group  and  mean 
individual  performance.  In  the  "limbs  only"  analysis,  the  group  value 
showed  a  210  (187)  utile  difference  and  the  corresponding  "mean" 
individual  function  showed  a  difference  of  87.5  (52.2)  utiles.  Hie  ratio 
between  group  and  "mean"  individual  performance  was  240%  (358%) . 

While  changing  the  structure  of  the  tree  seems  to  slightly  affect  the 
results,  the  sane  general  conclusions  can  be  made  with  regard  to  the 
assessment  of  the  completeness  of  subjects'  act  generation  performance. 
Subjects  do  not  generate  many  high  utility  acts.  In  fact,  their  ability  to 
generate  possible  acts  in  a  decision  situation  can  be  fairly  characterized 
as  "impoverished”. 


Conclusions 

Eapjinsu  trees,  and  Utility  estimation;  an  advance  in  ffletittfologfr. 

Hie  methodology  used  in  these  experiments  represents  an  advance  over 
earlier  techniques  used  to  study  act  generation.  Lorge,  Fox,  Davitz  and 
Brenner  (1358)  surveyed  a  number  of  studies  contrasting  individual  and 
group  performance,  some  of  which  were  concerned  with  act  generation.  Hiese 
studies  typically  evaluated  performance  by  counting  the  number  of 
responses  subjects  made.  Hie  more  recent  study  of  Pitz,  et.  al.  (1980) 
used  a  similar  technique. 

Hie  pooling  technique,  the  construction  of  hierarchical  act  trees,  and  the 
estimation  of  utility  of  acts  all  represent,  we  believe,  advances  in 
methodology  which  allow  the  researcher  to  address  issues  of  quality  as 
well  as  tymnm-y.  while  these  methods  have  problems,  such  as  the  effort 
required  to  implement  them,  they  may  prove  useful  in  other  contexts.  For 
example,  the  efficacy  of  various  structuring  techniques  used  in  decision 
analysis  could  be  examined.  Hiese  methods  should  also  prove  generally 
useful  for  the  study  of  predecision  processes.  Hie  availability  of  a 
hierarchical  act  tree  constructed  by  pooling  subjects'  acts  should 
facilitate  the  study  of  the  cognitive  processes,  such  as  memory  search 
strategies,  involved  in  predecisicn  processes.  Hiey  should  also  facilitate 
the  study  of  rules  a  decision  maker  uses  to  determine  whether  a  potential 
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act  should  be  considered  further. 

Generalizing  these  issuing  other  .contexts 

Problem  solving  tasks.  The  problems  employed  in  this  study  are 
representative  of  an  important  class  of  problems  in  which  the  goal  is 
fairly  well  defined,  but  the  steps  needed  to  reach  the  goal  are  not 
specified  (Taylor,  1974).  Most  problem-solving  research  has  been  confined 
to  problems  in  which  the  goal  and  the  steps  taken  to  reach  the  goal  are 
well  defined.  For  example,  in  chess  (de  Groot,  1965) ,  potential  acts 
involve  a  limited  set  of  moves,  and  the  problem  confronting  the  player  is 
to  arrive  at  the  best  sequence  of  acts.  Problems  in  which  the  possible 
acts  are  unspecified  are  more  carmen,  and  usually  are  more  important.  He 
suspect  the  lack  of  attention  to  prohlems  in  which  possible  acts  are  not 
specified  is  due  to  their  apparent  intractability.  Our  research  suggests 
that  the  intractability  is  more  apparent  than  real.  In  ary  event,  the 
results  reported  here  would  seem  to  be  relevant  to  problem  solving,  and 
should  not  be  considered  relevant  to  decision  theory  issues  alone. 

Expertise  revisited.  Earlier  a  distinction  was  drawn  between  an  expert  who 
performs  a  task  on  a  daily  basis  and  has  possible  acts  readily  accessible 
in  memory,  and  the  "generalist"  who  may  not  be  able  to  easily  access  acts 
in  memory.  In  our  past  research  on  hypothesis  generation  (Mehle,  note  5; 
Gettys,  et.  al. ,  note  4) ,  experts  and  nonexperts  attempted  to  generate 
possible  reasons  why  an  automobile  malfunctioned.  The  results  from  both 
populations  were  similar,  suggesting  experts  and  nonexperts  have  similar 
deficiencies  in  hypotheses  generation  due  to  the  operation  of  certain 
cognitive  mechanisms.  It  was  not  claimed  that  expertise  was  of  no  value; 
many  tasks  cannot  be  performed  unless  the  requisite  information  is  in 
memory.  It  is  too  early  to  knew  whether  similar  conclusions  can  be  drawn 
for  act  generation.  However,  the  magnitude  of  the  results  reported  here 
suggests  the  variable  of  expertise  should  be  investigated  in  act 
generation. 


Human  £&  generation  peffonaanCS  flBSSSBfifl. 

In  stannary,  these  experiments  address  two  major  questions,  one  dealing 
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with  the  completeness  of  act  generation  performance,  and  the  other  dealing 
with  the  cost  of  failure  to  generate  important  acts.  Subjects  appeared  to 
be  impoverished  act  generators,  suffering  both  in  the  aanyja  and  the 
quality  of  acts  generated. 
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Appendix  A 
A  PLACE  TO  LIVE 

Imagine  that  your  friend  is  a  new  OU  student  from  Canada.  He  was  able  to 
earn  and  borrow  enough  to  move  to  Norman  and  pay  his  tuition  and  fees  but 
he  does  not  have  enough  money  to  rent  an  apartment  or  live  in  the  dorms. 
In  fact,  his  monthly  check  front  his  parents  will  barely  pay  for  his  meals. 
Somehow  he  has  to  make  arrangments  for  a  place  to  live  for  the  rest  of  the 
semester. 

Ihe  US  Inmigration  Service  prohibits  foreign  students  from  working,  so 
suggesting  that  your  friend  get  a  job  to  earn  money  to  rent  an  apartment 
would  not  solve  the  problem.  Also,  your  friend  has  exhausted  his  credit  at 
Financial  Aids,  and  cannot  get  loans  from  any  other  source.  Your  friend 
will  not  do  anything  illegal  that  might  result  in  his  being  deported. 

Your  task  is  to  suggest  as  many  possible  solutions  for  his  problem  as  you 
can.  These  solutions  need  not  be  perfect  or  ideal,  in  fact  solutions  that 
have  definite  drawbacks  are  entirely  acceptable  as  long  as  they  have  the 
possibility  of  solving  his  problem  for  the  rest  of  the  semester.  Your 
solutions  to  this  problem  should  not  be  minor  variations  of  the  same 
solution. 

It  is  important  that  you  enter  all  new  options  which  occur  to  you.  This  is 
similar  to  ‘thinking  out  loud.'  We  do  not  want  you  to  censor  your  options 
and  only  enter  the  ones  which  you  think  are  particularly  good.  Put  down 
all  options  which  occur  to  you,  good  or  bad.  Your  score  in  this  experiment 
is  determined  by  the  number  of  options  you  generate  not  by  hew  good  they 


are 


APPENDIX  B 

ACT  STRUCTURE  FCR  PARKING  PROBLEM 
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Baaed  on  all  "logically  distinct"  acts,  PARKING  experiment 


14  *  1. 

(.37)  ** 

Use  aval: 

2 

(.07) 

1.1 

3 

(.07) 

1.2 

(.03) 

1.3 

2 

(.07) 

1.4 

2 

(.07) 

1.5 

3 

(.10) 

1.6 

(.03) 

1.7 

86  2. 
(.97) 

Increase 

40 

(.70) 

2.1 

(^7) 


1.2  Reduce  "non-parking"  space  in  lots 

1.2.1  do  away  with  "no  parking"  zone? 

1.2.2  do  away  with  extra  curb  space  in  lots 


1.5  Segregate  cars  according  to  size 

•ack  the  cars  in  more  tightly  by  painting 
closer  together 

1.7  Remove  30  minute  parking  meters 
Increase  available  space  for  parking 

2.1  Convert  more  University  land  into  parking 

2.1.1  At  the  dorm 

2.1.2  On  tie  South  Oval 

2.1.3  At  tie  Duck  Pond 

2.1.4  On  the  grass 

2.1.5  Convert  University  Streets  to 

2.1.6  At  Lloyd  Noble  (basketball)  St 

2.1.7  By  tearing  down  old  buildings 

2.1.8  By  parking  on  top  of  buildings 

2.1.9  By  parking  inside  the  football  stadiun 

2.2  Buy  or  use  non-University  land  to  create  more  parking 

2.2.1  Buy  adjacent  houses,  convert  to  lots 

2.2.2  USe  adjacent  shopping  center  at  night 

2.2.3  Reserve  streets  around  University  for  University 
parking 

2.2.4  Use  areas  around  fraternities  as  University  Parking 

2.2.5  Use  church  parking  lots 


26 

(.67) 


2.3  Build  parking  structures 

2.3.1  underground  stuctures 

2.3.2  high-rise  parking  structures 

2.4  All  future  buildings  should  be  high-rise  so  that  more  land 

around  them  will  be  available  for  parking 
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Si, 

6 

(.13) 


25 

(.53) 


59 

(.83; 


11 

(.23) 
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(.07) 


5 

(.17) 


(.30) 


3.  Reduce  demand  for  parking 


3.1  Restrict  number  of  cars  allowed  to  park 

3.1.1  by  lottery:  those  who  lose  ride  motor cyles 

3.1.2  allow  only  one  car  for  every  three  students 

3.2  Outlaw  cars 


3.2.1  make  everybody  rj.de  bikes 

3.2.2  make  everybody  ride  bikes  or  motorcycles 

3.2.3  for  faculty  and  staff  except  at  Lloyd  Noble 

3.2.4  except  for  cars  used  in  car pooling 

3.2.5  make  computer  students  park  at  Lloyd  Noble  and 

use  the  trolley  to  Main  Campus 

3.2.6  for  employees:  require  them  to  take  cabs 

3.2.7  except  for  commuters 

3.2.8  of  students  who  live  in  Norman 

3.2.9  of  Fraternity  members  from  Campus  and  have  them 

park  at  their  Prats 

3.3  Introduce  or  encourage  alternative  forms  of  transportation 

3.3.1  widespread  rapid- transit 

3.3.2  busing  for  surrounding  area 

3.3.3  encourage  mopeds,  bicycles,  and  motorcycles 

3.3.4  people  who  live  within  one  mile  are  asked  to 

walk  or  ride  bikes 

3.3.5  redbce  bicyle  thefts  to  increase  bike  riding 

3.3.6  build  bicycle  paths  to  increase  bike  riding 
.7  sell  lcw-pricea  bicycles 
.8  carpooling  for  students 

.9  provide  tr - — 

.10  improve  " 

.11  get  a  r< _ _ _ 

.12  store  dorm  residents 


i 


3. 


a  real  trolley  car  on  a  track 

■a  Amth  rocirlont-o'  rarR  Off— CSmpUS  with  bUS 


to  their  cars 

3.3.13  bus  service  to  shopping,  entertainment,  etc.  would 
reduce  need  for  cars 

3.3.14  promote  the  use  of  roller-skates 

3.3.15  build  sidewalks  and  improve  security  on  Campus 

3.4  Raises  or  incentives  for  employees  or  students  who 

3.4.1  don't  drive  to  school 

3.4.2  car pool 

3.4.3  n,ai  to  work 
3.4.3  ride  motorcycles 

3.5  Make  it  fashionable  not  to  drive  to  work 

3.5.1  Form  "Volksmarch"  clubs  to  encourage  walking 

3.5.2  Form  cycling  clubs 

3.6  Educate  people  about  comnuting  alternatives 

3.6.1  advertise  advantages  of  bikes  and  motorcycles 

3.6.2  educate  students  In  conserving  gasoline 

3.6.3  to  improve  enviornroental  quality 

3.7  Encourage  students  not  to  bring  their  cars  to  Canpus  from  home 

need  them 


3.7.1  unless  they 

3.7.2  when  they  ar  _ 

3.7.3  if  they  live  on  campus 


reall 
e  Frei 
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3.8  Build  moving  sidewalks 

3.9  Reduce  peak  demand  for  parking 

3.9.1  Schedule  classes  differently  to  reduce  demand 

3.9.2  Have  employees  work  alternate  days 

3.9.3  Have  empoyees  drive  only  on  alternate  days 

3.9.4  Let  commuters  plan  their  schedules  first  so  that 
they  can  car pool 

3.9.5  Make  commuters  stagger  class  schedules 

3.10  Increase  the  fees  for  parking  permits 

3.11  Build  more  on-campus  housing  for  students 

3.12  Offer  more  correspondence  courses  so  students  will  stay  at 

home 

3.13  Inpound  cars  of  all  violators  of  parking  regulations 

3.14  Raise  parking  fees  to  reduce  demand 


3.15  Build  Campus  in  direction  of  open,  unused  areas  so  more 
parking  will  be  available  xn  the  future 

4.  Change  parking  priorities 

4.1  Do  away  with  restricted  parking  (decaled) 

4.2  Let  people  park  anywhere  they  want  to 

4.3  Allot  a  specific  place  for  each  student 

4.3.1  based  on  seniority 

4.4  Reduce  service  vehicle  parking 

4.5  Reduce  faculty  parking  to  increase  student  parking 

4.6  Do  not  allow  GAs  to  park  in  faculty  parking 

4.7  Make  times  when  Faculty-Staff  parking  is  open  more  flexible  B2 

5.  Provide  faculty  and  staff  with  housing  within  walking  distance 

5.1  on  top  of  the  place  where  they  work 


61 

(.60) 

(.03) 


6.  Frivolous  and  other  responses 

6.1  Move  the  University  to  the  students  by  establishing  branch 
campuses 
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(.03) 

6 

(.20) 

4 

(.13) 

(.03) 

2 

(.07) 

(.03) 

7 

(.13) 


3 

(.07) 


7 

(.13) 

(.03) 

(.03) 

(.03) 

2 

(.07) 

(.03) 

3 

(.10) 


(.03) 

(.03) 

4 

(.03) 


6.2  Encourage  more  enrollment  of  foreign  students  (because  not 

as  many  own  cars) 

6.3  Reduce  student  population 

6.3.1  make  OU  an  "all  girls"  school 

6.4  Have  a  helicopter  pick  everybody  up  for  work 

6.5  Increase  the  enrollment  cost 


6.6  I  would  have  someone  towed  away  to  create  a  space  for  me 

6.7  Counterfeit  Faculty  parking  stickers 

6.8  Destroy  "useful”  areas 

6.8.1  tear  down  Owen  Field  (Football  Stadium) 

6.8.2  destroy  OU 

6.8.3  destroy  Frat  Houses 

6.9  Do  away  with  current  University  programs  to  create  parking 

6.9.1  Football 

6.9.2  Oklahoma  College  of  Continuing  Education 

6.10  Build  subway 

6.11  Destroy  cars 

6.11.01  threaten  to  destroy  cars 

6.12  Fire  teachers 


6.13  Take  away  parking  from  other  branches  of  the  University 

6.13.1  Post  Office  Training  Program 

6.14  Rambling,  incoherent  thoughts 

6.15  Let  only  people  who  have  paid  for  cars  drive 

6.16  Don't  let  people  who  attend  drive 

6.16.1  allow  only  walkers  to  attend  University 

6.16.2  make  everybody  live  on  campus 

6.17  Have  your  parents  drive  you  to  school 

6.18  Make  everybody  walk  who  has  less  than  a  2.3  GEA 


6.19  Impossible  acts 


ssue  everybody  $  pair  of  wings 
ssue  strength  pills 

t  than  to  school, (in  a  matter  transporter) 
reducing  or  shrinking  cars 


use 


(.03) 
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(.03) 

(.&) 


(.03) 

(.03) 

(.03) 

1 

1.03) 

(.fe) 

(.J3) 

(.03) 

(.03) 

(.03) 


6.20  Discourage  use  of  cars  by 

6.20.1  saying  that  the  Surgeon  General  has  found  cars 

cancerous 

6.21  Send  minorities  to  Central  State  University 

6.22  Iranian  (some  of  these  data  were  collected  when  the  American 
hostages  were  returned  from  Iran) 

6.22.1  Nuke  Iran 

6.22.2  Take  Iranians  hostage  until  they  give  OU  money  for 

parking  lots 

6.23  Let  only  ladies  drive 

6.24  Have  everybody  drive  golf  carts 

6.25  Ban  all  foreign  cars 

6.26  Have  computerized  lot  that  destroys  all  cars  exceeding  limits 

6.27  Use  area  we  have  now  for  parking  instead  of  building  the  new 

facilities  that  are  planned 

6.28  Have  harder  working  employees  so  you  don't  need  so  many 

6.29  Only  hire  people  in  wheel  chairs 

6.30  Only  hire  people  who  like  to  rim 

7.0  Responses  that  combine  two  or  more  options  from  above 


*  The  number  of  acts  in  each  limb  or  branch. 


The  proportion  of  subjects  who  gave  limb  or  branch. 


(  tree  with  act  numbers  on  June  81  qrt.  report  disk.  This  tree  on  July  81  TR  disk.) 
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APPENDIX  C 

Act  Structure  for  Living  Problem 
1.  Live  somewhere  without  paying  rent 

1.1  With  person  he  knows 

1.1.1  With  me 

1.1.2  With  friends 

1.1.3  With  my  parents 

1.1.4  Have  his  parents  move  here 

1.1.5  With  relatives  in  Norman 

1.2  With  person(s)  he  doesn't  knew 

1.2.1  In  fraternity  house 

1.2.2  With  perspn  of  opposite  sex 

1.2.3  Someone  with  extra  room  in  apartment 

1.2.4  Relatives'  friends 

1.2.5  With  experimenter 

1.2.6  With  religious  cult 

1.2.7  With  sponsor 

1.3  Public  institution 

1.3.1  Salvation  army 

1.3.2  Police  station 

1.4  Miscellaneous  places 

1.4.1  Tent 

1.4.2  Streets 


1.4o3  Oytside 

1.4.4  Mi  sc.  school  buildings 

1.4.5  Car 

1.4.6  Trash  dunpeter 

1.4.7  Qnpty  house 

1.4.8  Motel  lairtdry  room 

1.4.9  Extra  dorm  room 


1.5  Pay  for  room  only,  get  board  free 


2.  Exchange  goods  or  services  for  money  or  place  to  live 


2.1  He  could  exchange  services  for  place  to  live 


2.1.1  With  me 

2.1.2  With  someone. in  town 


2.1.3  Could  be  Res 


i&l' 


Assistant 


2.2  He  could  exchange  goods  or  services  for  money 

2.2.1  Sell  plasma 

2.2.2  Write  book  or  articles 

2.2.3  Sell  paintings  or  photographs 


l±i 


[n tings  or  photographs 

Sessions 
jpons  to  sell 


2.2.6  Clip  coupon j  to  sel 

2.2.7  Have  carnival 

2.2.8  Sing  or  juggle  in  si 

2.2.9  Collect  beer  cans 

2.2.10  Form  punk  rock  band 


juggle  in  street 
beer  cans 


2.2.11  Give  benefit  concert  for  self 

2.2.12  Enroll  in  work-study 
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104  3.  Obtain  money  through  other  sources 

(.90) 


33 

(.70) 


19 

(.33) 


3.1  Try  to  get  loan 


From  friends 
From  bank 
From  loan  shark 
Through  me 
From  me. 

From  relatives 
From  private  sponsors 
From  government 

3.2  Apply  for  financial  assistance 


m 

m 

m 


3.2.1  For.hoqsir 

1 12 From'  government 

3.2.4  From  school 

3.2.5  Enroll  as  foreign  exchange 


For  housing 

Apply  for  scholarship 

From  government 


33 

(.70) 


19 

(.27) 


3.3  Ask  for  money 

3.3.1  From  other  relatives 

3.3.2  From  my  parents 

3.3.3  From  his  parents 

Illll  P§:  ad  fr^paper  to  ask  for 

3.4  Donations 

3.4.1  Monetary  donations  from  me 

3.4.2  Of  living  quarters 

3.4.3  Fund-raising  projects 

3.4.4  Take  up  collection 


16 

(.27) 

9 

(.17) 


(.0?) 

(.03) 

(.0?) 


4.  Ask  someone  for  suggestions  or  help 


4.1  Someone  you  know 

i:B  £?Srr 

1.1.3  Relatives 
.1.4  My  parents 
4.1.5  Boyfriend's  mother 

4.2  School  officials 


4.3  Roommate  agency 

4.4  Conmuiity  organization 


5  5.  Try  to  get  regulations  changed 

( .17) 

5  5.1  on  working 

(.17) 


student 


contributions 
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30 

(.60) 

6. 

Change  current  plans 

2 

(.07) 

6.1  Wait  to  go  to  school  till  gets  more  money 

5 

(.17) 

6.2  Take  fewer  hours 

3 

(.10) 

6.3  Go  to  less  expensive  school 

(.03) 

6.4  Commute  from  Canada 

(.03) 

6.5  Cat  less,  cheaper  food 

(.03) 

6.6  work  in  Mexico  for  semester 

(.03) 

6.7  Break  law  so  will  be  deported 

14 

(.43) 

6.8  Go  home 

5 

(.13) 

7. 

Prepare  in  advance 

(.03) 

7.1  Apply  for  citizenship  149 

2 

(.07) 

7.2  Invest  money  134,135 

(.0^) 

7.3  Establish  residency  before  coming  146 

(.03) 

7.4  Pay  bills  so  have  good  credit  rating  148 

29 

(.40) 

8. 

Long  shots 

5 

(.13) 

8.1  Look  for  money 

8.1.1  Gold  49  _ 

8lII§  Witlwnetal^cletector  127 


9 

(.10) 


8.2  Steal  money 

8.2.1  Rob  bank  36 


8.3  Get  money  through  questionable,  illegal  methodB  37 


Assassinate  relatives, 98. 
Have  old  person  put  nils  in  w 
Sell  sister  to  white .slaver 
Get  hit  by  car.  sue  dri 
Jump  off  Dale  Hall  t 
Get  a  job  anyway  123 
Form  own  non-profit  oorp  155 


ii97 

_ver  121 
attract  attention  129 
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(.10) 


(.03) 

t.oi) 

4 

(.10) 

(.03) 

2 

(.07) 

(.03) 


8.4  Get  money  through  luck 

^.4.1  Sgre^cMld^f^om  drowning,  get  reward  131 

8.5  Pray  for  solution  to  problem  144 

8.6  Incur  revolution  147 

9.  Acts  which  will  not  solve  problem 

9.1  Conmit  suicide  150 


9.2  I  could  kill  him  152,154 

9.3  I  would  plead  guilty  to  his  illegal  activities  153 
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APPENDIX  D 

Tree  derived  from  complete  linkage  cluster  analysis  for  Parking  Problem 

31  1.0  Do  more  with  university-cwned  space. 

(.80)  ^ 

20  1.1  Structures  designed  to  aid  parking  problem 

1.1.1  Build  high  rise, 

1.1.2  Build  new  buildings  as  high  rises. 

11  1.2  Reorganize  current  parking  areas 

(.30) 


32 

(.53) 

5 

(.13) 


12 

(.37) 


4 

(.10) 


11 

(.20) 


36 

(.60) 

13 

(.43) 


certain  areas. 

1.2^3  Make  enouSTspaces  for  everyone "^Li^ng^n^^onns. 

1.2.4  Raise  parking  fees  to  improve,  expand  lots. 
Improve  condition  of  facilities. 

1.2.5  Remove  30  minute  meters. 

2.  Expand  parking  to  surrounding  areas 

2.1  Create  more  parking  space  in  surrounding  areas 

2.1.1  Put  parking  si< 

2.1.2  Builalots  : 

2.1.3  Look  for  avi _ 

new  facilities. 

2.2  Build  more  lots 

2.2.1  Expand  Lloyd  Noble  parking  facility, 
d  present  areas. 

__  more,  lots. 

_ id  another  lot  like  Lloyd  Noble. 

Build  more  places  like  i-lcyd  Noble. 

2.2.4  Build  a  new  lot  by  the  30CX  dorms. 

2.3  Change  parking  status  in  certain  areas 


ling 


2.3.1  Make  some  faculty  parking  student  parking. 

2.3.2  Regulate  staff  parking  hours  more  closely. 

2.3.3  Use  0U  vehicle  parking  for  students. 


2.4  Use  other  areas  for  parking 

2.4.1  Don't  have  so  many  no  parking  zones. 

M  A  A  V  _  . _ _ ^  _ _ I  _  _  1  M _ If.  _ 1 


2.4.4  Use  ar enii  around  frats  for  overflew  parking. 

2.4.5  Use  church  lots. 

2.4.6  Make  frat  members  park  at  their  own  houses. 
3.  Use  alternate  forms  of  transportation 


3.1  Carpooling 

3.1.1  Encourage  carpooling  by  canmuters. 

3.1.2  0U  could  organize  a  car pool. 

3.1.3  Provide  greater  incentives  for  carpooling. 
Reduce  price  stickers  for  car pools. 

3.1.4  Ask  conmuters  to  car pool. 

Use  carpools. 
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17 

(.37) 


34 

(.77) 

6 

(.13) 


28 

(.73) 


3 

(.10) 


97) 


3.2  Encourage  people  not  to  drive. 


1.2.1  Enoour. 

1.2.2 


age  people  not  to  bring  cars, 
il  students ^about  parking  problem.. 


;e  more  Knowledge ~abo&t  cxpraSuKng  available, , 

courses  available. 


3.2.3  Have  more  selection  of  evening  < 
3.3  Promote  other  forms  of  transportation. 


3.3.1  Advertise  the  advantages  of  riding  a  bike 
or  motorcycle. 

More  students  could  ride  bikes. 

3.3.2  Form  cycling  clubs  for  commuters. 

3.3.3  Encourage  students  to  stop  pollution  by 
riding  Dikes. 

Ask  all  students  to  ride  bikes. 

3.3.4  Build  bike  paths. 

3.3.5  Provide  more  security  for  bikes. 


4.0  Park  elsewhere ,  get  other  transportation  in. 


4.1  Force  members  of  university  community  to  park  elsewhere. 

4.1.1  Mandatory  parking  for  faculty,  staff  at  Lloyd  Noble. 

4.1.2  Have  employees  park  at  Lloya  Noble,  use  trolley. 

4.1.3  Have  commuters  park  at  Lloyd  Noble,  use  trolley. 

4.1.4  Make  all  commuter  parking  at  Lloyd  Noble. 

4.2  Encouracje  parking  in  other  areas  by  making  rapid  transit 


4.2.1  Set  up  rapid  transit  system  during  school  hours. 
Set  up  bus  system  for  surrounding  area. 

Set  up  bus  system  for  surrounding  communities. 
Set  up  mass  transit  system  for  Norman, 

4.2.2  Enact  bus  system  from  campus  to  shopping  malls. 

4.2.3  Improve  trolley  and  CART  systems. 

Provide  more  trolleys  and  busses. 

Provide  transportation  from  South  campus. 

5.  Impound  cars  without  stickers. 


6.  All  acts  having  zero  or  lower  utility  which  were  not 
a  part  of  the  cluster  analysis 


(On  disk  146) 
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lems  was  studied  by  a  series  of  three  experiments.  In  the  first  experiment, 
subjects  were  given  a  realistic  decision  problem  and  asked  to  respond  with 
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20.  (continued) 

any  act  occurring  to  them.  In  the  second  experiment,  the  acts  suggested  were 
evaluated  by  different  subjects  for  feasibility.  In  a  third  experiment,  ad¬ 
ditional  subjects  estimated  the  utility  of  the  acts  judged  feasible.  The 
act  generation  performance  of  subjects  was  evaluated  using  two  techniques. 
First,  a  decision  tree  was  generated  by  the  experimenters  by  combining  the 
acts  suggested  by  all  subjects.  The  decision  tree  generated  by  each  subject 
was  compared  with  the  experimenter- generated  tree.  It  was  found  that  sub¬ 
jects  failed  to  generate  important  limbs  and  branches  of  the  group  decision 
tree.  Second,  the  quality  of  the  trees  generated  by  individual  subjects 
was  evaluated  by  an  opportunity  loss  calculation.  This  calculation  provides 
an  estimate  of  the  potential  cost  of  failing  to  generate  limbs  and  branches 
of  the  decision  tree.  The  opportunity  loss  analysis  suggested  that  the 
failure  to  generate  a  complete  tree  could  be  costly. 
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